Student Experiences on Taking Electronic Exams at the University of Helsinki

Various forms of electronic exams have been provided at the University of Helsinki for years, but all in small scale; the mainstream is still to take a lecture hall exam written with pen on paper. Two pilot projects – the online exam pilot at the faculty of social sciences and the electronic examining project – provide electronic exam opportunities for teachers and students. This case study is based on student experiences on electronic examining in the pilot projects, collected by electronic questionnaires. The results indicate that online and electronic exam room exams may lower stress related to examining, promote deep learning, provide flexibility in studies as part of busy life and thus promote to shorter graduation times and increase the quality of learning. On Electronic Examining An exam can be defined as “an on purpose created problem situation included in education” of which the student has to manage with his/her knowing. The exam situation includes measurement, external control, achievement assessment, linkage to real life and requirements from the society (Karjalainen 2001). In comparison to other methods of passing courses, examining often refers to summative assessment. In summative assessment the results are used to grade students at the end of a course or at the end of a programme. Thus, summative assessment takes place after teaching is concluded. Formative assessment refers to giving feedback to students during the learning process (Biggs & Tang 2007). In this paper, an exam is defined as a summative assessment method with purpose to give a final grade to students. To create the opportunity for students to take exams, an exam process is needed. In the process, when described on a coarse level, the teacher defines what the student is expected to produce, the student registers for the exam, produces the required outcome and the teacher assesses the outcomes and registers the course grade in the study register system. When making the exam electronic, the change refers to the student part of the examining process; computers and networks are used in producing the required outcome. In an electronic exam, student responses can be automatically assessed, such as in multiple choices, calculations, pieces of programming code or database queries. Departments of Computer Science have provided such exam and practice systems for years, e.g. SQL Trainer at the University of Helsinki (Laine 2001), Trakla2 at Aalto university among others. Web-based learning environments such as Moodle provide opportunities for automatically assessed question types. In this paper, however, the exam situations refer to writing essay responses for open-ended questions. They are assessed by the teachers by hand. Automatic assessment for essay responses has been discussed and tested, but in essays the language plays a significant role compared to fixed-form question types. Therefore the idea is at development stage even concerning the internationally used English language. So for the Finnish language used on limited geographical area, such idea would still take time to mature. Additionally, assessment engines may require teaching with large numbers of responses (Kakkonen 2003) and Finnish with a challenging grammar would make it even more difficult to create such an assessment engine. In all, the essay assessment systems are more pilots and tests than real systems that could be used in public. When investigating on the exam situation in more detail, there are two variables that restrict the situation: time and the physical exam space. When these are combined, four ways of providing electronic exams are defined by the authors, as illustrated in Table 1. [1] http://www.cse.hut.fi/en/research/SVG/TRAKLA2/(16.4.2014) [2] http://docs.moodle.org/26/en/Question_types (16.4.2014) [3] http://www.kaggle.com/c/asap-aes (17.4.2014) Is the physical space restricted? Yes No Is the exam time restricted? Yes · Exams in computer classrooms · Bring Your Own Device exams · Online exams No · Electronic exam room exams · Online assignments Table 1: Electronic exams in time and space, by the authors. When both the time and space are restricted, the students are expected to be present at a specific time in a specific place. There, the exam situation reminds of a traditional exam in a lecture hall written with pen on paper. But instead of pen and paper, a computer is used. The exam situation is invigilated in the traditional way by one or more exam invigilators. When only the physical space is defined but students are allowed to select any time suitable for them for taking the exam, then the examining method is called electronic exam room examining. There the university provides the facilities, including a room with computers and electronic monitoring. The student uses a booking system on the web and makes a reservation for the exam. When the student has taken the exam, the teacher can see and assess the response in the system. When the physical space is not restricted, the exam types are called online examining. If the time is not limited, the specification online exam period can be used. There, the concept of examining gets close to making assignments online, and even the technical alternatives are close to each other or even the same. In open-book, open-web examinations the students are allowed to use material, web, notes, scrap paper in producing their answers. The different forms of electronic exams can be compared to traditional examining and each other in terms of scalability, flexibility and other added values. Exams taken in physical rooms are restricted by the number of seats in the room. In that sense, online exams are scalable from small to large numbers of students, based on needs. On the other hand, the restricted time in online exams may be a challenge for some students that perceive added value from the flexibility that electronic exam rooms provide. Indirect added values include shorter study times and higher quality as developed learning outcomes. Compared to traditional examining, all forms of electronic examining are written on computers which may be a value, even an added value, e.g. for accessibility reasons. Traditional examinations often represent closed-book method which means that students are not allowed to have any study material or notes with then in the examination. Open-web examination, on the other hand, represents open-book method where the students can have study materials available in the examination. One factor that affects students’ performance in examinations, and that traditional examination often causes, is test anxiety (Dochy & McDowell 1997). Test anxiety, consisting of affective (physiological arousal, emotionality), cognitive (worry), and behavioral (procrastination, avoidance) components, weakens academic performance (see Hembree 1988; Cassady & Johnson 2001). Thus, we assess in particular students’ experiences of stress and test anxiety in these two types of examinations. Electronic Examining at the University of Helsinki At the University of Helsinki, there have been electronic exam rooms for ten years. Computer classrooms have been used for electronic examining, as well. At least as long there have been possibilities for students to take online exams. None of these have risen to the major examining method, though; instead, traditionally supervised exams written with pen on paper are most often used at the University of Helsinki in 2014. To activate changes in the situation, the faculty of social sciences started the online exam pilot in 2009, and the university-wide Electronic examining project started in 2013. These pilot projects aim at developing electronic examining as part of educational development, and as such, collect data from students and teachers on their experiences on different forms of electronic examining. Thus, the projects aim at emphasizing added values provided by the various forms of electronic examining. The Electronic examining project is spit into subprojects of which one is to equip and pilot a larger room than the existing rooms for one or two students. The new room includes 16 computers and video monitoring. There are ca 15 voluntary teachers from five faculties who pilot their exams in the electronic exam room or are going to, since the pilot is ongoing. Of the pilot teacher group, three teachers from the faculty of social sciences have already provided altogether seven exams. [4] http://www.helsinki.fi/valtiotieteellinen/opkeh/online_exam.html (17.4.2014) [5] http://wiki.helsinki.fi/display/sahkoinententti/ (16.4.2014) The web-based system which is used in the pilot is of own production from 2005 and is very simple in features but also in user interface. The idea is that first the teacher creates an exam and adds a number of questions in the system. The questions are added in groups of at least five parallel alternatives of which at least one is randomly drawn for each student. For example, if the teacher wants the students to answer to three questions, she has to add 3x5=15 questions in the system. When the exam is published, the students can book time slots for the exam. Times are available according to the building opening hours. When the time comes, the student goes into the room, logs in the system, writes the answers and submits. After that, the teacher receives an automatic email message about the submission and is able to assess it with grade and feedback. Instructions and rules for use are published on the web. The exams in the online exam pilot at the faculty of social sciences are provided in the web-based learning environment Moodle which is widely used in teaching and studies at the University of Helsinki. All students get familiar with Moodle in their ICT Driving license course they take during their first study year, and additionally, to verify that students know how to submit the exam, they are allowed to practice the submission process in a practice area some days before the exam. Online exams are scheduled on Saturdays during the same time the traditional faculty exams are taken. Therefore they are always 4h in time. The examination questions were delivered and responses uploaded using a course area in Moodle. The examination was a fourhour’s synchronous session, but students could also work offline during the exam time. Besides the open-book method, the online examinations were without invigilation, and the students could take the examination wherever they had access to Internet, for instance at home, in a library or in a learning center. During the examination, helpdesk support by phone was available for students seeking assistance and participants could also practice in advance how to submit responses. Thus, the type of online examination the faculty of social sciences was piloting corresponded to the faculty examinations in terms of scope, text books and time limit. They also contributed to students’ degree the same ways as other examinations. The differences were the place, free use of resources and that the examinations were without invigilators (Joutsenvirta 2009).